Speech recognition programs may operate on devices having various capabilities. A client device may offload speech recognition tasks to another device having more robust capabilities by transferring speech recognition data to the other device. Determining which tasks to offload and when to offload tasks can involve balancing computing resource and availability factors with recognition accuracy, user experience, and other considerations.